Edge enhancement system and method and imaging system

ABSTRACT

A plurality of bandpass filters extract high frequency components of a digital image signal sampled every predetermined unit, thus generating a plurality of high frequency signals. The high frequency signals are added to produce a resultant high frequency signal. On the basis of the value of the resultant high frequency signal, the maximum value or minimum value detected from the high frequency signals is selected. The resultant high frequency signal and the selected maximum or minimum value are added according to determined weights, so that an enhancement signal is generated. Edge enhancement is performed using the enhancement signal.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.10/982,751 filed Nov. 3, 2004 which claims priority from Japanese PatentApplication No. 2003-377118 filed Nov. 6, 2003, both of which the entirecontents are hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an edge enhancement system and methodfor performing edge enhancement of image signals.

2. Description of the Related Art

FIG. 5 is a block diagram of a conventional edge enhancement circuit513. In FIG. 5, a luminance-signal generation circuit (MainY) 501generates a main luminance signal derived from image data on which edgeenhancement is to be applied. A horizontal bandpass filter (H-BPF) 502detects a horizontal edge component of the main luminance signalgenerated from the luminance-signal generation circuit 501 to generate ahorizontal edge signal. A gain circuit 505 applies an arbitrary gain tothe horizontal edge signal generated from the horizontal bandpass filter502 to control the amplitude of the signal.

Similarly, a vertical bandpass filter (V-BPF) 503 detects a verticaledge component of the main luminance signal to generate a vertical edgesignal, and a diagonal bandpass filter (D-BPF) 504 detects a diagonaledge component of the main luminance signal to generate a diagonal edgesignal. Gain circuits 506 and 507 control the amplitudes of thecorresponding edge signals, respectively.

Adders 508 and 509 add the horizontal, vertical, and diagonal edgesignals to generate an edge enhancement signal. A gain circuit 510controls the amplitude of the edge enhancement signal. Finally, an adder511 adds the main luminance signal generated from the luminance-signalgeneration circuit 501 to the edge enhancement signal generated from thegain circuit 510, thus generating an edge-enhanced luminance signal.

In the conventional edge enhancement circuit 513, the horizontal,vertical, and diagonal edge signals are added, thus generating aresultant edge signal. When the amplitudes of the respective edgesignals are large, the amplitudes overlap each other. Thus, theamplitude of the resultant edge signal is too large. This can result inthe degradation of quality of an image subjected to edge enhancement. Inthis instance, a spatial frequency domain with the problem of amplitudeoverlap resulting in abnormally large amplitude will now be described.FIG. 6 shows regions occupied by the horizontal, vertical, and diagonaledge signals in the spatial frequency domain. Referring to FIG. 6, theabscissa denotes a horizontal spatial frequency axis and the ordinatedenotes a vertical spatial frequency axis. As a region comes closer tothe origin, its frequency becomes lower. Referring to FIG. 6, regions Hare detected by the horizontal bandpass filter 502, regions V aredetected by the vertical bandpass filter 503, and regions D are detectedby the diagonal bandpass filter 504. A horizontal edge signal overlaps adiagonal edge signal in the region shown by the arrow a of FIG. 6. Thus,the above-mentioned problem occurs.

FIG. 7 shows an example of the generation of a resultant edge signalhaving an abnormally large amplitude in the region shown by arrow a ofFIG. 6. Referring to FIG. 7, the horizontal bandpass filter 502generates a horizontal edge signal H having a waveform 701 in the regionshown by the arrow a of FIG. 6. The diagonal bandpass filter 504generates a diagonal edge signal D having a waveform 702 in the regionshown by the arrow a of FIG. 6. In this case (the amplitude of avertical edge signal is zero), as shown in FIG. 7, the maximum amplitudeof a resultant edge signal Mix having a waveform 703 is too largecompared to a predetermined value obtained by superimposing theamplitude of the diagonal edge signal on that of the horizontal edgesignal. The disadvantage is that unnatural edge enhancement is caused bythe above resultant edge signal (enhancement signal) having theabnormally large amplitude.

SUMMARY OF THE INVENTION

The present invention is made in consideration of the above-mentionedproblems by providing an edge enhancement system and method capable ofpreventing unnatural edge enhancement in images such as complex videoimages.

According to an aspect of the present invention, an edge enhancementsystem for images is disclosed. The edge enhancement system includes: ahorizontal bandpass filter for extracting high frequency signals of asampled digital image signal in a horizontal direction; a verticalbandpass filter for extracting high frequency signals of a sampleddigital image signal in a horizontal direction in a vertical direction;an adder for adding the high frequency signals generated from thehorizontal bandpass filter and the vertical bandpass filter to produce afirst high frequency signal; a maximum value detector for detecting themaximum value of the high frequency signals for each sampling to producea second high frequency signal; a minimum value detector for detectingthe minimum value of the high frequency signals for each sampling toproduce a third high frequency signal; and a weight adder fordetermining weights for the first, second, and third high frequencysignals based on the value of the first high frequency signal and addingthe first high frequency signal to the second or third high frequencysignal according to the weights to produce a fourth high frequencysignal.

According to another aspect of the present invention, an edgeenhancement method is disclosed. The edge enhancement method includes:extracting high frequency signals, by a horizontal bandpass filter, of asampled digital image signal in a horizontal direction; extracting highfrequency signals, by a vertical bandpass filter, of a sampled digitalimage signal in a horizontal direction in a vertical direction;extracting horizontal high frequency signals of a digital image signalsampled in a horizontal direction; extracting vertical high frequencysignals of a digital image signal sampled in a vertical direction;adding the horizontal high frequency signals and the vertical highfrequency signals generated to produce a first high frequency signal;detecting the maximum value of the high frequency signals for eachsampling to produce a second high frequency signal; detecting theminimum value of the high frequency signals for each sampling to producea third high frequency signal; and determining weights for the first,second, and third high frequency signals based on the value of the firsthigh frequency signal and adding the first high frequency signal to thesecond or third high frequency signal according to the weights toproduce a fourth high frequency signal.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention.

FIG. 1 is a schematic block diagram of an edge enhancement circuit 413according to an embodiment of the present invention.

FIG. 2A shows output signal waveforms of components of the edgeenhancement circuit 413 shown in FIG. 1.

FIG. 2B shows output signal waveforms of components of the edgeenhancement circuit 413 in FIG. 1.

FIG. 3 is a diagram showing the input-output characteristic of aweighted additional coefficient calculation circuit included in aweighted average circuit 116.

FIG. 4 is a schematic block diagram of an imaging system having an edgeenhancement circuit in accordance with an embodiment of the presentinvention.

FIG. 5 is a block diagram of a conventional edge enhancement circuit513.

FIG. 6 shows a spatial frequency domain including regions occupied byhorizontal, vertical, and diagonal edge signals.

FIG. 7 shows an example of the generation of a resultant edge signalhaving an abnormally large amplitude within the region of arrow a ofFIG. 6.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described in detail inaccordance with the accompanying drawings. An imaging system in which anedge enhancement circuit is employed will now be described.

FIG. 4 is a schematic block diagram of an imaging system having an edgeenhancement circuit in accordance with an embodiment of the presentinvention. In FIG. 4, a lens 401 and an optical filter 402 form anobject image on an imaging device 403. The optical filter 402 includesan optical low-pass filter and an infrared cut filter. The imagingdevice 403 includes a CCD (charge coupled device) sensor or a CMOS(complementary metal oxide semiconductor) sensor. In this case, theimaging device 403 has RGB (red, green, and blue) color filters.

An analog-to-digital (A/D) converter 404 converts the analog signaloutput from the imaging device 403 into a digital signal (image data).The image data is subjected to dark noise correction by an optical blackcircuit (OB) 405 after which a white balance circuit (WB) 406 performswhite balance processing on the image data generated from the opticalblack circuit 405. A color interpolation circuit 407 performsinterpolation calculation for each color pixel of the image data outputfrom the white balance circuit 406. A color conversion matrix (MXT)circuit (ColorMTX) 408 converts the image data (R, G, and B signals),subjected to interpolation calculation and generated from the colorinterpolation circuit 407, into YUV data. The YUV data consists of a Ysignal representing the luminance signal, a U signal, which is thedifference between the luminance signal and the R component, and a Vsignal, which represents the difference between the luminance signal andthe B component.

A chroma suppress circuit (CSup) 409 processes the YUV data to suppressthe colors of chroma components within a saturation region. A UV gaincircuit (UVGain) 410 applies arbitrary gains to the U and V signalsincluded in the YUV data generated from the chroma suppress circuit 409,respectively.

In addition, the image data generated from the white balance circuit 406is also supplied to a luminance-signal (Y-signal) generation circuit(MakeY) 412. The Y-signal generation circuit 412 performs interpolationcalculation on the image data to produce a luminance signal. An edgeenhancement circuit (Enhance) 413 uses the produced luminance signal toperform edge enhancement processing (e.g., edge emphasis).

A luminance-signal generation circuit (MainY) 101, shown in FIG. 1,generates a main luminance signal based on a luminance signal generatedfrom the Y-signal generation circuit 412 in FIG. 4. A horizontalbandpass filter (H-BPF) 102 detects a horizontal edge component of themain luminance signal generated from the luminance-signal generationcircuit 101 to generate a horizontal edge signal. A vertical bandpassfilter (V-BPF) 103 detects a vertical edge component of the mainluminance signal of the luminance-signal generation circuit 101 togenerate a vertical edge signal. A diagonal bandpass filter (D-BPF) 104detects a diagonal edge component of the main luminance signal of theluminance-signal generation circuit 101 to generate a diagonal edgesignal.

A gain circuit 105 applies an arbitrary gain to the horizontal edgesignal generated from the horizontal bandpass filter 102 to control theamplitude of the signal and then generates an amplitude-controlledhorizontal edge signal Yh. Similarly, a gain circuit 106 applies anarbitrary gain to the vertical edge signal generated from the verticalbandpass filter 103 to control the amplitude of the signal and generatesan amplitude-controlled vertical edge signal Yv. A gain circuit 107applies an arbitrary gain to the diagonal edge signal generated from thevertical bandpass filter 104 to control the amplitude of the signal andgenerates an amplitude-controlled diagonal edge signal Yd.

An RGB conversion circuit (ConvRGB) 411 converts the U and V signalsgenerated from the UV gain circuit 410 and the luminance signalgenerated from the edge enhancement circuit 413 into RGB data. A gammacorrection circuit (GammaRGB) 414 performs gamma correction on the RGBdata generated from the RGB conversion circuit 411. A JPG conversioncircuit (JPG) 415 converts the RGB data generated from the gammacorrection circuit 414 into JPEG (Joint Photographic Experts Group)image data. The JPEG image data is then recorded onto a recording medium(not shown) that is detachable from the imaging system.

First Embodiment

First, the schematic structure of an edge enhancement system included inan imaging system (for example, like a digital camera) according to anembodiment of the present invention. FIG. 1 shows an edge enhancementcircuit according to an embodiment of the present invention. This edgeenhancement circuit 413 in FIG. 1 is employed in the imaging system ofFIG. 4.

A luminance-signal generation circuit (MainY) 101 generates a mainluminance signal based on a luminance signal generated from the Y-signalgeneration circuit 412 in FIG. 4. A horizontal bandpass filter (H-BPF)102 detects a horizontal edge component of the main luminance signalgenerated from the luminance-signal generation circuit 101 to generate ahorizontal edge signal. A vertical bandpass filter (V-BPF) 103 detects avertical edge component of the main luminance signal of theluminance-signal generation circuit 101 to generate a vertical edgesignal. A diagonal bandpass filter (D-BPF) 104 detects a diagonal edgecomponent of the main luminance signal of the luminance-signalgeneration circuit 101 to generate a diagonal edge signal.

A gain circuit 105 applies an arbitrary gain to the horizontal edgesignal generated from the horizontal bandpass filter 102 to control theamplitude of the signal and generates an amplitude-controlled horizontaledge signal Yh. Similarly, a gain circuit 106 applies an arbitrary gainto the vertical edge signal generated from the vertical bandpass filter103 to control the amplitude of the signal and generates anamplitude-controlled vertical edge signal Yv. A gain circuit 106 appliesan arbitrary gain to the diagonal edge signal generated from thevertical bandpass filter 104 to control the amplitude of the signal andgenerates an amplitude-controlled diagonal edge signal Yd.

An adder 108 adds the horizontal edge signal Yh to the vertical edgesignal Yv. An adder 109 adds the output of the adder 108 to the diagonaledge signal Yd to generate a resultant edge signal Ymix. In other words,the adders 108 and 109 add the horizontal edge signal Yh, the verticaledge signal Yv, and the diagonal edge signal Yd shown in the followingExpression 1 to produce the resultant edge signal Ymix.

Ymix=Yh+Yv+Yd  (Expression 1)

For every pixel, a maximum value detection circuit 110 detects themaximum value from values of the horizontal edge signal Yh, the verticaledge signal Yv, and the diagonal edge signal Yd to generate a maximumsignal Ymax. A minimum value detection circuit 111 detects the minimumvalue from values of the horizontal edge signal Yh, the vertical edgesignal Yv, and the diagonal edge signal Yd every pixel to generate aminimum signal Ymin. A gain circuit 112 controls the amplitude of themaximum signal Ymax. A gain circuit 113 controls the amplitude of theminimum signal Ymin. When the edge signal Ymix is positive, a selectioncircuit 114 selects the amplitude-controlled maximum signal Ymaxgenerated from the gain circuit 112 and outputs the selected signal as aselection signal Ysel. When the edge signal Ymix is negative, theselection circuit 114 selects the amplitude-controlled minimum signalYmin generated from the gain circuit 113 and outputs the selected signalas the selection signal Ysel.

A subtractor 115 subtracts the selection signal Ysel from the edgesignal Ymix to produce a difference signal Ydiff. According to thepresent embodiment, the difference signal Ydiff indicates the absolutevalue of the difference between the edge signal Ymix and the selectionsignal Ysel.

A weighted average circuit 116 applies weighting and averaging to theedge signal Ymix and the selection signal Ysel on the basis of thedifference signal Ydiff to produce an enhancement signal Yenhance.Specifically, to calculate a weighted additional coefficient α (0≦α≦1)on the basis of the difference signal Ydiff, the weighted averagecircuit 116 includes a weighted additional coefficient calculationcircuit (not shown) having the input-output characteristic shown in FIG.3. The weighted additional coefficient calculation circuit calculatesthe weighted additional coefficient α to determine the ratio of the edgesignal Ymix to the selection signal Ysel to be added on the basis of thedifference signal Ydiff. As shown in FIG. 3, as the difference signalYdiff becomes larger, the weighted additional coefficient calculationcircuit obtains the weighted additional coefficient α so that thecoefficient α approaches 0. As the difference signal Ydiff becomessmaller, the weighted additional coefficient calculation circuitcalculates the weighted additional coefficient α so that the coefficientα approaches 1. The weighted average circuit 116 calculates thefollowing Expression 2 using the edge signal Ymix, the selection signalYsel, and the weighted additional coefficient α which is obtained basedon the difference signal Ydiff, thus producing the enhancement signalYenhance.

Yenhance=Ysel×αYmix×(1−α)  (Expression 2)

In other words, when the difference signal Ydiff is large, the weightedaverage circuit 116 determines that the edge signal Ymix has anabnormally large amplitude in a region of the spatial frequency domainwhere the outputs of the bandpass filters overlap each other as shown bythe arrow a of FIG. 6 and uses Expression 2 and the weighted additionalcoefficient α=1.0 to obtain the enhancement signal Yenhance. Since theaddition ratio of the selection signal Ysel is large and the additionratio of the edge signal Ymix is small, the enhancement signal Yenhancecan be produced without being influenced by the edge signal Ymix havingthe abnormally large amplitude. When the difference signal Ydiff issmall, the weighted average circuit 116 determines that the amplitude ofthe edge signal Ymix is small in the region of the spatial frequencydomain where the outputs of the bandpass filters do not overlap eachother or even where the outputs of the bandpass filters overlap eachother, and calculates Expression 2 using the weighted additionalcoefficient α=0.0 to obtain the enhancement signal Yenhance. In thiscase, the addition ratio of the selection signal Ysel is small and theaddition ratio of the edge signal Ymix is large in the enhancementsignal Yenhance, resulting in effective edge enhancement using the valueof the edge signal Ymix.

The input-output characteristic of the weighted additional coefficientcalculation circuit is not limited to that shown in FIG. 3. Thefollowing input-output characteristic may be used: As the differencesignal Ydiff becomes larger, the weighted additional coefficient αapproaches 1. As the difference signal Ydiff is smaller, the weightedadditional coefficient α approaches 0.

The operation of the edge enhancement circuit 413 in FIG. 1 will now bedescribed with reference to FIGS. 2A and 2B. FIGS. 2A and 2B showexamples of output signal waveforms of the edge enhancement circuit 413in FIG. 1. The output signal waveforms are generated when a luminancesignal in, for example, the frequency region shown by the arrow a ofFIG. 6 is processed by the edge enhancement circuit 413. Note that theluminance signal is obtained by capturing a vertically striped object,and the output Yv of the vertical bandpass filter 103 indicates zero. InFIGS. 2A and 2B, the luminance signal and various signals obtained byprocessing the luminance signal are digital. Each digital signalconsists of discrete values sampled every pixel.

In FIG. 2A, the horizontal edge signal Yh has a signal waveform 201. Thehorizontal edge signal Yh is generated from the gain circuit 105, whichcontrols the amplitude of the output signal of the horizontal bandpassfilter 102. The diagonal edge signal Yd has a signal waveform 202. Thediagonal edge signal Yd is generated from the gain circuit 107 thatcontrols the amplitude of the output signal of the diagonal bandpassfilter 104. Referring to FIG. 2B, the resultant edge signal, Ymix is acombination of Yh, Yv and Yd and also has a signal waveform 203.Specifically, the edge signal Ymix is generated by adding the horizontaledge signal Yh (signal waveform 201) and the diagonal edge signal Yd(signal waveform 202) by using the adders 108 and 109.

Referring to FIG. 2A, the maximum signal Ymax has a signal waveform 204.The maximum signal Ymax is output by the maximum value detection circuit110 based on inputs from Yh and Yd (signal waveforms 201 and 202,respectively). The minimum signal Ymin has a signal waveform 205. Theminimum signal Ymin is output by the minimum value detection circuit 111based on inputs from Yh and Yd (201 and 202). The maximum signal Ymax,subjected to amplitude control through the gain circuit 112, has asignal waveform 206. The minimum signal Ymin, subjected to amplitudecontrol through the gain circuit 113, has a signal waveform 207.

When the edge signal Ymix is positive, the selection circuit 114 selectsthe signal waveform 206. On the other hand, when the edge signal Ymix isnegative, the selection circuit 114 selects the signal waveform 207. InFIG. 2B, the selection signal Ysel has a signal waveform 208. Theselection signal Ysel is obtained by combining the maximum and minimumvalues of the signals generated from the respective bandpass filters.

The subtractor 115 generates the difference signal Ydiff indicating theabsolute value of the difference between the signal waveform 203 of theedge signal Ymix and the signal waveform 208 of the selection signalYsel. Subsequently, the weighted average circuit 116 performs weightingand averaging on the edge signal Ymix and the selection signal Ysel onthe basis of the difference signal Ydiff to produce the enhancementsignal Yenhance having a signal waveform 209 shown in FIG. 2B. When theedge signal Ymix with the signal waveform 203 has an abnormally largeamplitude, the enhancement signal Yenhance can be equal to orapproximate to the value of the selection signal Ysel with the signalwaveform 208, which has a normal amplitude.

Other Embodiments

The imaging system is not limited to that shown in FIG. 4. So long asedge enhancement can be performed on an image signal such as a luminancesignal and on RGB signals, any imaging system within the spirit andscope of the present invention may be used. An object to be subjected toedge enhancement is not limited to a luminance signal. RGB signals and acomplementary color signal can be subjected to edge enhancement.Although shown with three filters, the edge enhancement circuit 413 ofFIG. 1 can employ two or more filters. The gain circuits are arranged invarious locations in the edge enhancement circuit 413 in FIG. 1. Ifamplitude control is not needed, any gain circuit can be omitted.

Furthermore, after the program read from the storage medium is writtento a function expansion board inserted into the computer or to a memoryprovided in a function expansion unit connected to the computer, a CPUor the like mounted on the function expansion board or functionexpansion unit performs all or a part of the actual processing so thatthe functions of the foregoing embodiments can be implemented by thisprocessing.

As many apparently widely different embodiments of the present inventioncan be made without departing from the spirit and scope thereof, it isto be understood that the invention is not limited to the specificembodiments thereof except as defined in the appended claims.

1. An edge enhancement system comprising: a horizontal bandpass filterfor extracting high frequency signals of a sampled digital image signalin a horizontal direction; a vertical bandpass filter for extractinghigh frequency signals of a sampled digital image signal in a horizontaldirection in a vertical direction; an adder for adding the highfrequency signals generated from the horizontal bandpass filter and thevertical bandpass filter to produce a first high frequency signal; amaximum value detector for detecting the maximum value of the highfrequency signals for each sampling to produce a second high frequencysignal; a minimum value detector for detecting the minimum value of thehigh frequency signals for each sampling to produce a third highfrequency signal; and a weight adder for determining weights for thefirst, second, and third high frequency signals based on the value ofthe first high frequency signal and adding the first high frequencysignal to the second or third high frequency signal according to theweights to produce a fourth high frequency signal.
 2. The systemaccording to claim 1, wherein when the first high frequency signal has apositive value, the weight adder adds the first and second highfrequency signals according to the weights, or when the first highfrequency signal has a negative value, the weight adder adds the firstand third high frequency signals according to the weights, thusproducing the fourth high frequency signal.
 3. The system according toclaim 1, further comprising a diagonal bandpass filter for extractinghigh frequency signals of a sampled digital image signal in a horizontaldirection, wherein the adder adds the high frequency signals generatedfrom the horizontal bandpass filter, the vertical bandpass filter, thediagonal bandpass filter to produce a first high frequency signal;
 4. Anedge enhancement method comprising: extracting high frequency signals,by a horizontal bandpass filter, of a sampled digital image signal in ahorizontal direction; extracting high frequency signals, by a verticalbandpass filter, of a sampled digital image signal in a horizontaldirection in a vertical direction; extracting horizontal high frequencysignals of a digital image signal sampled in a horizontal direction;extracting vertical high frequency signals of a digital image signalsampled in a vertical direction; adding the horizontal high frequencysignals and the vertical high frequency signals generated to produce afirst high frequency signal; detecting the maximum value of the highfrequency signals for each sampling to produce a second high frequencysignal; detecting the minimum value of the high frequency signals foreach sampling to produce a third high frequency signal; and determiningweights for the first, second, and third high frequency signals based onthe value of the first high frequency signal and adding the first highfrequency signal to the second or third high frequency signal accordingto the weights to produce a fourth high frequency signal.